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Abstract — Motivated by applications such as recoimnendation 
systems, we consider the estimation of a binary random field 
X obtained by unknown row and column permutations of a 
block constant random matrix. The estimation of X is based 
on observations Y, which are obtained by passing entries of X 
through a binary symmetric channel (BSC) (representing noisy 
user behavior) and an erasure channel (representing missing 
data). We analyze an estimation algorithm based on local 
popularity. We study the bit error rate (BER) in the limit as the 
matrix size approaches infinity and the erasure rate approaches 
unity at a specified rate. Our main result identifies three regimes 
characterized by the cluster size and erasure rate. In one regime, 
the algorithm has asymptotically zero BER, in another regime 
the BER is bounded away from and 1/2, while in the remaining 
regime, the algorithm fails and BER approaches 1/2. Numerical 
results for the Movielens dataset and comparison with earlier 
work is also given. 

I. Introduction 

Recommendation systems are commonly used in e- 
commerce to suggest relevant content to users. One approach 
considers the user-item rating matrix, predicts the missing 
entries, and recommends items based on the predicted values 
(for example, see H]). Recently, a number of researchers have 
considered mathematical models for this problem and studied 
fundamental limits. One model assumes the rating matrix to be 
a low-rank random matrix ( Ill-El), and then bounds on the 
number of samples needed to recover the complete matrix with 
high probability are obtained. In another model ( ||5l, JH), the 
rating matrix is assumed to be obtained from a block constant 
matrix by applying unknown row and column permutations, 
a noisy discrete memoryless channel representing noisy user 
behavior, and an erasure channel denoting missing entries. The 
goal for such a model is not matrix completion, but estimation 
of the underlying "noiseless" matrix. In ||5], ||6l, the probability 
of error in recovering the entire matrix for fixed erasure rate 
is considered, and threshold results reminiscent of the channel 
coding theorem (but with different scahng) are established. 

In this paper, we consider the model in ||6l, but we allow 
the erasure rate to approach unity, and focus on the BER 
- the probability of error that a specific recommendation 
fails. We analyze the BER for a specific algorithm, which 
makes recommendations based on "local popularity". Such an 
analysis is of interest for two reasons: 

• It gives an upper bound on achievable BER; 

• The local popularity algorithm used is motivated by 



algorithms used in practice Q, and has lower complexity 
compared to those in the above mentioned references. 
• The algorithm has competitive empirical performance on 
real datasets such as the Movielens data fSl. For example, 
next we compare the algorithm with OptSpace ||3l on 
Movielens data. While OptSpace uses ratings on the scale 
1-5 given by Movielens, in our algorithm we quantize 
the ratings as follows: 4,5 are mapped to 1, while 1-3 
are mapped to 0. (Similarly, the output of OptSpace is 
quantized to {0, 1}.) We find that the local algorithm 
yields a BER of 0.091, while on the same test data, 
OptSpace gives a BER of 0.107. Thus the performance 
of both algorithms is similar. (More detailed simulation 
results will be presented in a future publication.) 
In this paper, we seek to understand the reason for the com- 
petitive performance of the relatively simple local algorithm 
by analyzing its BER for the model proposed in |[5|. Suppose 
that the matrix is of size n x n and the erasure probability 
e = 1 — c/n". If a e [0,1/2), then our main result says 
that if the cluster size is greater than n"^'^" where 7„ — > 0, 
then the BER approaches 0, but if the cluster size is less than 
7i" \ 7 > 0, the BER is bounded away from zero and a 
lower bound is obtained in terms of the observation noise and 
7. For a > 1/2, BER always approaches 1/2. Due to space 
constraints we only provide an outline of the proofs; the details 
with additional results will be reported in a journal submission. 

The rest of the paper is organized as follows. In Section 
nil we describe our model, the local popularity algorithm, and 
establish notation. The main results are stated and discussed 
in Section Uni The proof of the main result is given in IIVI and 
some related lemmas are established in |V] The conclusion of 
given in Section IVll 

II. Basic Setup 

In Section III-AI we describe our model, and discuss a local 
popularity based algorithm in Section III-BI 

A. The Model 

We consider an n x n rating matrix X whose entries are 
binary. The rows of the matrix represent users and the columns 
represent items. Suppose A = and B = {Bi}^^^ are 

row and column partitions respectively, representing sets of 
similar users and items. We assume that for all i = 1, . . . , r 
we have \Ai\ = \Bi\ — k. The sets Ai x Bj are the clusters 



of the matrix and they are unknown. If (p, q) G AiX Bj, then 
X(p, g) = where are i.i.d. Bernoulli(l/2). This matrix 
X is passed through a memoryless binary symmetric channel 
(BSC) with parameter p, and then through an erasure channel 
with each entry being erased independently with probability e. 
The erasures characterize the missing entries in a rating matrix, 
while the BSC characterizes the noisy behaviour of the users. 
The entries of the observed matrix Y are from {0, 1, *}, where 
* denotes an erased entry. 

We consider the case of binary entries and uniform cluster 
size is for simplicity, and like in |f6l, these can be relaxed. For 
more detailed motivation of this model, we refer to ||5|, 

B. A Local Popularity Algorithm 

Without loss of generality suppose the first row belongs to 
Ai. Upon observing Y, we want to recommend an item (a 
column) to the user 1. In this paper we study a particular 
"local" algorithm, which only uses pairwise row correlations. 
Let the number of commonly sampled entries between two 



(Small cluster size) If there is a constant 7 > such that 
k < n""'', then 



rows (similarity) Si 



{Y(»,fc)5^*} ■ l{Y0-,fc)5^*} • 

l{Y(i,A:)=Y(j,fc)}' where 1{ j denotes the indicator function. 
We use the following local algorithm (local_algo(T)) to 
recommend an item Jq to user 1. 



local_algo (T) : 

1) (Select the top T nearest rows) Compute su, for 
i = 1,2, ... ,n. Select the top T rows with the highest 
values of similarity, where T is a parameter whose 
choice is discussed later. 

2) (Pick the most popular column) Among the columns 
j such that Y{l,j) — *, select the column having 
maximum number of I's among the top T neighbors. 
Break ties randomly. 



Suppose we represent each row by a vertex in a graph 
with an edge between vertex i and j iff Sij > 0. Then 
to recommend an item to user 1, the above algorithm 
depends only on the rows neighboring to user 1, and 
chooses the most popular item among the top few neighbors. 
Hence we use the adjective "local popularity". We study 
the probability of error for this algorithm, denoted as 
Pe[local_algo(r)] Pr[X(l,jo) = 0]. 

III. Main Result 

From the results in |6J, it follows that for k > cin" \ogn, 
a E [0, 1/2), with high probability we can recover the entire 
matrix X using a "local" algorithm, and hence the BER also 
approaches zero. In the following theorem, we establish a 
stronger result for local_algo(T). 

Theorem 1. Suppose a e [0,1/2) and c > 0. Assutiie that 
the erasure probability e = 1 — the BSC error probability 
p G [0, 1/2), and r goes to infinity with n. 

• (Large cluster size) If there exists a sequence 7„ > 
such that 7„ — > and k > n°'~'^", then 
Pe[local_algo{k)] — > n — > 00. 



lim inf Pe[local_algo{k)] > 



P 



)UJ + (1 -p)UJ 



In Theorem[T]we restrict ourselves to a G [0, 1/2). For a < 
1/2, as we show in Section [TVl all the rows picked by Step 1 
of the algorithm are from Ai ("good") with high probability. 
But, for a > 1/2, most of the rows picked are from outside Ai 
("bad"), and hence the algorithm breaks down. Due to lack of 
space, the results for a > 1/2 will be presented in subsequent 
publications. In the rest of this paper, we present a proof of 
Theorem [T] 

IV. Proof of Theorem[T] 

In this section we present the proof of Theorem [T] To begin 
with, we introduce some notation. 

Notation: By X ^ B{n,p) we mean that a random vari- 
able X is binomially distributed with parameters n and p. 
For a real valued function /(n), by ri(/(n)), 0(/(n)) and 
o{f{n)) we represent the standard asymptotic order notation 
(see for example ||9] p. 433]). We say that f{n) = g{n) if 
lim„^oo = 1- For a matrix X, X(:,j) denotes the jth 
column of X. For a vector y € {0,1,*}", |y|o, \y\i and \y\ 
represent number of O's, number of I's and the total number 
of O's and I's respectively. For a sequence of events {En], if 
P[En] 1 with n, then we say that £"„ occurs w.h.p.. 
Analysis of Step 1 of the algorithm: We show that w.h.p. the 
top k rows are all from Ai. We observe that for i G yli\{l}, 
si, - B{n,pg) with pg := (1 - ef[(l - pf + p\ For 
i ^ ^1 we observe that si ; is a mixture of binomials with 
= npb for ph ^ ^' < pg. We omit the proofs 
of the following two lemmas, which are consequences of 
the Chemoff bound ifTOl Theorem 1.1] together with a union 
bound. 

Lemma 1 ( Overlap with "good" rows). For 5 <E we 

have 

Pr [ min su < npg(l - S)] < ke'^'P^^^/^ =: pi. 

Lemma 2 (Overlap with "bad" rows). For 6 G (0, 1), we 

have 

Pr \ maxsii > npi,(l+5)'^] < {n—k)e 5 +2re^~6~ ='■ P2- 

Since pg > pi,, we can choose a small enough constant 6a 
such that npg{l — 6q) > npi,{l + 5q)'^ . Let Ei denote the event 
that there is an error in Step 1 of the algorithm, i.e., we choose 
some rows from outside Ai in the top k users. Using Lemma 
[T| and Lemma |2] we obtain 



Pr[Ei] < Pr 



min Su < maxsi^ 



<Pl+P2 = o{l). (1) 



Here (a) follows since npg — Q{npb) ~ 0(n ), and r 
increases to infinity with n. This implies that w.h.p. Step 1 of 
local_algo does not contribute to the error 



Analysis of Step 2 of the algorithm: We assume that Step 
1 picks all the k "good" neighbors, (i.e., we condition on the 
event Ef.) 

Large cluster size: Suppose k > n"^^" for 7„ 

0(1). Let jmax denote the most popular column chosen by 
local_algo(fc), and suppose and Yfc denotes the ma- 
trices X and Y respectively, restricted to the top k rows. Since 
we have conditioned on Ef, we observe that for a column j 
such that X(l,j) = 1, we have |Yfc(:,7)|i - B{k, (l-e)(l- 
p)). Define fiy := E[|Yfe(:, j)|i] and ^ V ar {\Y ki:, 
to obtain the following two lemmas. 



Pe[local_algo(A:)] = Pr[X(l, j) = Q\j^ax = j] 



(a) 



Pr[X(l,j) =0b. 



^Pr[X(l,j) = 0, Yfe(:,j) = y\j„,a. - .hE^] + o(l) 



p\v\i-\v\o 

~ yela.i,*}'' pIS'li^l^'l" + (1 ~ p)\y\i-\y\o 
yeM 



0(1) 



(3) 



Lemma 3 (Many I's in the most popular column). For 

different values of k, we have the following lower bounds on 

\^ k (• ; jmax ) 1 1- 

\) If k = n°'~'*" such that 7„ > and 7„ — >■ 0, then w.h.p. 

\'yk{-,jviax)\i > min{vTogn, =: ii(ri). 
2) If k = n'^gnfor g„ > 1, then w.h.p. \Yki:, j„iax)\i > 

max{/iy + min{fTy A/Iogn}(Ty, \/Togn} =: t2{n). 

Proof: The proof is given in Section IV-AI ■ 

Lemma 4 (I's form majority in the most popular col- 
umn). Let jmax be the most popular column. Then w.h.p. 
|Yfc(:, j,„aj;)|i - \yk{-;jjnax)\o increases to 00 with n. 

Proof: The proof is given in Section IV-BI ■ 
Now we use Lemma [3] and Lemma |4] to prove that the 
local algorithm makes vanishingly small probability of error. 
We define t{k,n) ti{n) if k — n"^''" for 7„ — > 0, and 
t{k,n) := t2{n) if fc = n"g„ for (7„ > 1 (here ti{n) and 
{t2{n) are as defined in Lemma O. Suppose 

M e {0,1,*}''^ : (|y|i-|y|o) -> 00, and \y\i >t{k,n)}. 

We also observe that for a column j, 

^k{-,j) > Yfe(:, j) > {j,nax = j}, (2) 

i.e., the random variables {Xk{:,j),Yk{:,j),{jrnax = j}} 
form a Markov chain. We are interested in finding the overall 
probability of error. In the following, by pk,j{y) we mean 
Pr[Yfe(:, j) = y\jmax = j, Ef]. Then we have 



where (a) follows from ([T]i, (b) is true because of Lemma |3] 
and Lemma |4] (c) is due to the Markov property (|2]i and the 
notation of Pkjiv), (d) is the Bayes' expansion, and (e) is true 
since for y S M, |y|i — \y\o goes to infinity with n, and the 
fact that p.v^^i_.p)^ = o{x) for p < 1/2. This proves the first 
part of Theorem [1] 

Small cluster size: Now suppose k < for a constant 

7 > 0. We show that in this case the most popular column 
has a finite number of unerased entries. This allows us to find 
a lower bound on the probability of error. 

Lemma 5 (Finite number of unerased entries). W.h.p. 

max|Yfe(:,j)|< L1/7J- 

3 

The proof is based on bounding the tail of Yfe(:, j) and is 
not given here due to space restrictions. Suppose 

I -.^ {ye {0,1,*}" -.m < L1/7J}- 

We want to find a lower bound for the total probability of 
error. By following the steps as in (O and replacing the event 
M by the event / (this replacement is justified due to Lemma 
|5]l, we have 



Pe[local_algo(A:)] 



E 



|y|i-|y|o 



ye{o,i,*} 



^ p\y\i-\y\B + (1 — p)\y\i-\y\o 



S|-Pfcj(2/) + 0(1) 



\y\i-\y\o 



> min — r-rr ,_, , ,,-1 i-i 
~ ye{o,i.*}^' plj/li-lylo + (1 - p)\y\i-\y\o 
y'ei 



0(1) 



(a) 
> 



P 



pL'J + (1 -_p)UJ 



o(l) 



y<^{o,i,*} 

J2 Pr[X(l,j)-0|Y,(:,j) = y,£;j].p,,,(y) + o(l) 

y&lo.i.*}" 
yeM 



where (a) is trues since |^|i — |y|o < \y\ < LVtJ for V ^ E 
and for a; G M, p^^^i_py is a decreasing function of x for 
p < 1/2. Taking liminf to both the sides proves the claim. 

V. Proofs of lemmas 

To prove Lemma [3] and Lemma |4] we need the following 
theorem. Suppose Q{t) denotes the upper tail of a standard 
normal distribution, i.e., Q(t) := —^= f°° e^* ^^dt. 

Theorem 2 (Moderate deviations for binomial distribu- 
tion). Suppose X„ ^ B{n,pn). If t„ ^ 00 in such a way 
that t^ — o {yar{Xn)) ~ o(rip„(l — Pn)), then 



id) 



E 



i/e{o,i.*}' 
yeM 



E 



i/e{o,i,*} 

yeM 



Pr[Yfc(:,j) = y|X(l,j) = 0,ga 
2Pr[Yk{:,j)^y\El] 

_p)l!/lo 

plsli (1 — p)\y\a + plslo^i — p)\y\i 



Pk,jiy) + o{l) 



Pfcj(y) + 0(1) 



Pr[Xn > npn + tn\/npn{l ~ Pn)] ^ Q{tn)- 

The above theorem is an adaptation of a theorem about 
moderate deviations of binomials when p„ is a constant ifTTl 
p. 193]. The proof is very similar to the one presented in ifTTI 
for the constant probability case, and is omitted here due to 
lack of space. 



A. Proof of Lemma \3\ 

1) Recall that we have conditioned on the event that all the 
rows in the top k neighbors chosen by local_algo(fc) are 
"good". Suppose k = n""''". Let S be the set of columns 
j such that X(l,j) = 1. Thus IS*] - B{n,l/2) and due to 
Chernoff bound we have w.h.p. \S\ > n/3. For a column 
j € S we see that \Yk{:,j)\i ~ B{k, (l-e)(l-p)), and they 
are independent for different values of j. Thus for j E S, 

Pr[|Yfc(:,.7)|i ><] > Pr[\Yk{:, ^ t] 



> 



> 



c(l-p)V 



-2i„(2)c/„-'"^ for large 



(4) 



> [ C{l-p) \ ^_21n(2)c 



where (a) is true since 1 — (1 — e)(l — p) > e, (b) follows 
since e = 1 - c/71", l-x> e-2i"(2)^ for x S [0, 1/2], and 
(t) > (f )* (see m p. 434]), and (c) is true because 7„ > 0. 
Since w.h.p. \S\ > n/3, we now have 



Pr[\Yk{:,Jrnacc)\l < t] 

<Pr 



max|Yfc(:,j)|i <i |5| >n/3 

•1/3 



< 1- 



£(l_lP)^ g-21n(2)c 



„ / c(1-_p1\'„-2 1„(2)c 

<e-3l-F;rT^j + o(l) 



+ 0(1) 

0(1) 



Suppose we put t = to := min{-y/log n, 2^}- Then 



c(i-rt 



(c(l-p))' 



c(l-p) 



/log n 



(5) 



o(n), 



where (a) follows since jnt < 1/2 and t < -y/log n. Thus from 
(|5]l we obtain 

Pr[\Yk{:,j,nax)\i < to] < e'^ + o(l) = o(l). 

This proves the first part of the lemma. 

2) Recall that we have assumed k = n"g„ for .g„ > 1. By 
following a very similar analysis as in the first part, we see 
that w.h.p. \Yk{:,j,nax)\i > Vlog n. In particular for .g„ = 1 
(or equivalently for k ^ n"), ^ becomes 



Pr[\Yk{:,j)\i>t] > 



kV ( C(l -p)Vg-21„(2)c 



c(l-p) 



t 



-21n(2)c 



(6) 



Observe that for two random variables X and Y such that 
X ^ B{ni,p) and Y ~ B{n2,p) with rii > 712, we have 
Pr[X >t]>Pr[Y>t]. Thus using Q we have 

Pr [|Yfe(:,j)|i > > l] > [|Yfe(:,i)|i > t\gn = l] 



> 



C(l -P) A ^-21„(2)c 



Hence for i = ^/log n, Q has the following counterpart, 

Pr[\Yk{:,j,na.)\i<t]<e-^i^)^ + o{l) 

= +0(1) 0(1). 

But in LemmaHlwe need better bounds for gn — > 00, and we 
consider this case now. Recall that for j £ S, fiy = E[|Yfc(: 
>i)|i] = c(l -p)gn and erf, = Var{\Yk{:,i)\i) = .g »c(l - 
p)(l — (1 — e)(l — p)). We define t„ := minjay y^Iogn}. 
Since ay — > 00, we have <fj = o(o'i')' and then Theorem |2] 
implies that for a column j e 5, 



(a) 

Fr[|Yfc(:,j)|i > /iy + <„ay] =Q(t„) = 

1 1 
>- 



1 



'2 ^/2^^„ 



(b) 



17 



27rt„ 

^2 /o 

e , for large 

1 



•y/n log n 



where (a) is true because Q(t) ^ ,^ e * ifTTl Lemma 

V 27rt 

1.2], and (b) is true since t„ < -y/log n. Since w.h.p. \S\ > 
n/3, we have 



Pr[\Yki:,j 
<Pr 



\l< ^J■Y + tnC^y] 



max|Yfe(:, j)|i < fiy + tn(7y\\S\ > n/3 
i6S 



o(l) 



< 1-17 



1 



\/n log n 



i/3 



+ 0(1) =0(1). 



Thus w.h.p. |Yfc(:, j™Qa.)|i > + t„cry, if .g„ ^ o o- We 
have already observed that w.h.p. \Yk{:, jmax)\i > \/Iogn. 
Thus the lemma is implied. 

B. Proof of Lemma |4] 

Lemma |3] gives us a lower bound for \Yk{:, jrnax)\i that 
holds w.h.p.. Next we find an upper bound for \Yk{:, jmax)\o 
to prove Lemma |4] 

First we condition on the event that ^.{l, jmax) = 1- We 
observe that 



|Yfc(:,j)|o^ |Yfc(:,j-)h 



Then conditioned on the value of |Yfc(:, j„taa;)|i = t, the 
distribution of \Yk{:, jmax)\Q does not depend on the fact that 
jmax is the most popular column chosen by the algorithm, and 
hence \Yki:,j,nax)\Q ^ B {k - t,po), where po := p{ilZ)+e - 
This is because for a given column j of Y^., upon observing 
that there are exactly t I's, the other k — t entries are i.i.d. 
with probability of being po- 

1) Suppose k ~ n" such that 7„ — > 0. We define 
b{k,p,i) := (*^)p'(l — p)"^* to be the zth binomial term, and 
observe that b{k,p,i) < {kpe/if, since ( ) < {ke/iy (see 



in p. 434]). We see that 



Pr 



|Yfc(:,j. 



> 



^ b{k-t,po,i) 



21os 



^ b{k-t,pQ,i) 



k-t 

E 

z— 2 log n+1 



<2\ogn-bik-t,po, 



(b) 

<21og7i 



<21ogn 



(fc - t)poe 
VlSg^/2 
2c' 



2 



^fc - t) 



b{k - <,_po,«) 
•5(fc-t,po,21ogn + l) 

/, ,N \ 21ogn+l 
(fc - t)pQe 



2 log n 
c' 



1 



n')'"(21ogn + 1) 



21ogn+l 



o(l). 



where (a) is true since b{k,p,i) is a decreasing function of 
i for i more than fcp and we have (fc — t)pQ = o(l), (b) is 
due to the fact that b{k,p,i) < {kpe/i}^ , and (c) follows by 
observing that kpoe < for a constant c' > 0. Thus 

w.h.p. we have \Yki:, j„iax)\o < 



Now suppose 7„ > 



Then we see that 



Pr 



|Yfe(:,j„ 



(a) 



fe-t 



< ^ b{k-t,po,i) < J2 iik-t)poe/iy 




where (a) is true since b{k,p,i) < {kpe/iy, (b) follows 
because kp^e < (^^^^ for a constant c', (c) is true by 
observing that for x ~ o(l), we have X^i^m ^ 0(a;™), 



and (d) follows since 



< 



whenever 7„ > 



47„ ^ 2 /ft - 2v^Iogrr' 

Thus we have proved that w.h.p. \Yk{:,j,nax)\Q < 
iiim{^^^^, j^}. This together with the observation in 
Lemma |3] that w.h.p. \Yk{:Jmax)\i > minj^logn, 5^}, 
proves that w.h.p. \Yk{:J,nax)\i - \Yk{-, jmax)\o increases 
to 00 with 71. 

2) Now we consider the other case of fc = n°'gn for 
9n > 1. If gn is upper bounded by a constant, then arguments 
very similar to those used in the first part tell us that w.h.p. 
\Yki:,jmax)\a < VTogn/2. 

In the remaining part of the proof, we assume that (7„ — ?> 00. 
Recall that for a column j such that X(l,j) = 1, we have 
A^Y =E[|Yfc(:,j)|i] =fc(l-e)(l-p) and af, = Far(|Yfc(: 
, = fc(l-p)(l-e)(l-(l-e)(l-p)). Conditioned on the 
value of \Yh{:, jmax)\i — t, suppose fiy and cr|> denote the 



conditional mean and variance of \Yk{:, jmax)\o- We observe 
that for t > fiY and large enough n, 

= {k- t)po < iiy, and ct|> = (fc - i)po(l - Po) < Sciy. 

Suppose tn :~ minjcTy^"', \/\-Og n}. Since ay — > 00, we have 
*n = o((t|,), and since w.h.p. yi \Y jniax)\i > Mr (see 
Lemma [3]), using Theorem |2] we obtain 



Pr 



<Pr 



)|o > My + ycry 



|Yfe(:,j- 

\Yk{:,jmax)\o > f^Y + 
0(1). 



o(l) 



tn 

.2^/2 

Thus w.h.p. \Yk{:,j,nax)\o < max{Vlogri/2,/iy + |^cry}. 
This together with the observation made in Lemma [3] that 
w.h.p. \Yk{:,jmax)\i > max{Vlogn, cry}, proves that 

w.h.p. \Yk{:,jrnax)\i~ \Yk{:, jmax)\o incrcascs to 00 with n. 
Remark: In the above proof, we had conditioned on the event 
that X{l,j„,ax) = 1- When we condition on X{l,jrnax) = 0, 
we have po = (iip)('il7)+e ' ^'^'^ ^ similar set of steps prove 
the claim. 



VI. Conclusion 

We have considered estimation of a binary random field 
obtained by permuting rows and columns of a block constant 
matrix, by observing a sub-sampled and noisy version. It 
would be interesting to analyze the performance of "local" 
algorithms on a more general class of matrices obtained from 
realizations of a "smooth" stochastic process. Further, non- 
uniform sampling models are also of interest. 
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